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O ■ Abstract 

The magnetic proximity effect is a fundamental feature of heterostructures composed of layers 
^ , of topological insulators and magnetic materials since it underlies many potential applications 

in 

t^^ ' in devices with novel quantum functionality. Within density functional theory we study 

(N ■ 

magnetic proximity effect at the 3D topological insulator/magnetic insulator (TI/MI) interface 
C^ ■ in Bi2Se3/MnSe(lll) system as an example. We demonstrate that a gapped ordinary bound state 

cn 

which spectrum depends on the interface potential arises in the immediate region of the interface. 
• T— I The gapped topological Dirac state also arises in the system owing to relocation to deeper atomic 

X' 

H ■ layers of topological insulator. The gap in the Dirac cone is originated from an overlapping of 

C^ ■ 

the topological and ordinary interfacial states. This result being also corroborated by the analytic 
model, is a key aspect of the magnetic proximity effect mechanism in the TI/MI structures. 

PACS numbers: 73.20.-r, 75.70.Tj, 85.75.-d 



To efficiently realize the potential of three dimensional (3D) topological insulators 
(TIs) in spin electronic and magnetic storage applications, it is advisable to integrate 
the TI layers into hybrid heterostructures containing the layers of ferromagnetic (FM) or 
antiferromagnetic (AFM) materials [l|, l^]- One has to tune these heterostructures in such a 
way that the spectrum of the 3D TI surface electron states should be easily accessible to an 
exchange field influence of FM (AFM) layer without significant spin dependent scattering 
of these states on magnetic ions. At the same time, the spin dependent transport of carriers 
in 3D TI should be controllable. These demands are not easily feasible because of a number 
of obstacles. On the one hand, a serious problem is to find the FM or AFM material which 
forms a high-quality interface with the TI material and at the same time provides a strong 
magnetic interaction with it. On the other hand, the physics of the exchange coupling at 
the TI/FM (AFM) boundary is not yet well understood. In principle, there exist different 
possibilities to provide an exchange field influence from a magnetic material on the surface 
electron states of a 3D TI. 

One way is to use the effect of the surface magnetic order in 3D TIs with chemisorbed 
magnetic impurities. While the local magnetic moments of impurities are arranged inside 
a thin layer with the thickness of the order of their diffusion length into TI and form 
magnetically ordered overlayer, the region of spin polarization of carriers near the 3D TI 
surface may be significantly larger due to magnetic proximity effect. This type of order 
can be realized by deposition of magnetic ions of 3d-metals (Mn, Fe, Cr, Co) on Bi2Te3, 
Bi2Se3 or Sb2Te3 |3|-l6|. When the surface concentration of ions is relatively high an indirect 
exchange coupling among their local magnetic moments mediated by the surface states of 
3D TI can arise (7|. As a result the system becomes unstable with respect to FM order 
with a spontaneous magnetization along the normal axis, which is accompanied by opening 



a gap in a spectrum of the Dirac surface states |7Hll|. However, if the energy of exchange 
coupling is smaller than the reciprocal lifetime of the TI surface state due to impurity 
disorder scattering, both the FM ordering and the energy gap should be suppressed. 

Another way to induce magnetic order on the surface of 3D TI's is coating with an 
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where FM order was 



external FM or AFM overlayer. This has been done in Ref. 
induced in Bi2-xMn2;Te3 (a;=0.09) by a magnetic proximity effect through the deposited 
Fe overlayer in the temperature range well above the intrinsic Curie temperature of the 
bulk Bi2_a;Mna.Te3. Unfortunately, in the case of metallic FM (AFM) materials the TI 



surface states near the TI/FM(AFM) interface should be significantly altered due to their 
hybridization with the bulk states of FM (AFM) metal. While DFT calculations are absent 
for this type of structures, one can suppose that the spectrum of the TI surface states should 
even lose helical features due to the alternation, since the density of states of the metallic 
FM (AFM) overlayer is much larger than the density of surface states of the 3D TI. 

The film of traditional FM (AFM) insulator (below called magnetic insulator, MI) 
adjacent to TI is a most promising candidate to manipulate the helical states of 3D TI by 
means of magnetic proximity effect [l3|. Such a way may diminish the surface scattering via 
continuum states of the magnetic film, contrary to the metallic film case. Recently, several 
Mis (EuO, EuS, EuSe, MnSe, MnTe, RbMnCls) with compatible magnetic structure and 
relatively good lattice matching with TIs (Bi2Te3, Bi2Se3 or Sb2Te3) are identified, and the 



best candidate material is found to be the large gap AFM semiconductor MnSe |14|. 

In the present paper, we study the physics of magnetic proximity effect at the TI/MI 
interface within first-principles DFT calculations for the Bi2Se3/MnSe(lll) system as an 
example. We show that two types of interfacial bound states (referred as the topological 
and interfacial ordinary state, respectively) appear at the TI side of the interface. These 
states have different physical origins, spatial distributions and energy spectra. Namely, the 
topological state stems from a breaking of the Z2 invariant of TI at the boundary with MI. 
This state is located relatively distant from the interface plane; its spectrum is gapped and 
lies inside the bulk energy gap of TI. In contrast, the interfacial ordinary state results from 
the crystal symmetry breaking at the TI/MI interface. This state is located nearby the 
interface and is strongly spin polarized, its spectrum is gapped and lies far below the bulk 
energy gap of TI due to the band bending at the TI side of the interface. 

For structural optimization and electronic bands calculations we use the Vienna Ab Initio 



Simulation Package [15|, |16| with generalized gradient approximation (GGA) [17| to the 



exchange correlation potential. The interaction between the ion cores and valence electrons 

nn 

was described by the projector augmented- wave method p^, 119|. The Hamiltonian contains 
scalar relativistic corrections, and the spin-orbit interaction (SOI) is taken into account by 
the second variation method [20(]. To describe correctly the highly correlated Mn-rf electrons 
we include the correlation effects within the GGA+U method as developed in Ref. |2l| . 

To simulate the Bi2Se3/MnSe(lll) heterostructure, the in-plane lattice constant of the 
MnSe is fixed to that of Bi2Se3. The most stable structure of bulk MnSe is a cubic NaCl-tj^e 
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FIG. 1. (Color on-line) (a) Crystal structure of the Bi2Se3/MnSe(lll) fcc-type interface (half of the 
supercell is shown); (b) Atomic charge difference with respect to the electronic charge of bulk-like 
atoms; (c) Total electrostatic potential averaged over xy planes Vxy (red line) and change of the 
potential with respect to that in the central parts of the MnSe (AVmi, light blue line) and 612863 
(AVti, dark blue line) slabs. Dashed line shows Boltzmann fit for AVti. Vertical black lines mark 
position of atomic layers; vertical green lines denote borders of Bi2Se3 QLs; z = corresponds to 
the interfacial plane. 



lattice, with antiferromagnetic ordering along the [111] direction 22|. The optimization of 
MnSe with the fixed parameter in (111) plane leads to 10% contraction of Mn-Se interlayer 



distance in the [1 
and J = 1.0 eV 



■ 1] direction. The typical values of correlation parameters U = 5.0 eV, 
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24| are appropriate for cubic MnSe, while for distorted MnSe they 
give Mn d states within the gap. For this reason U = 6.0 eV, repelling Mn d bands into 
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FIG. 2. (Color on-line) Electronic structure of Bi2Se3/MnSe heterostructure. Size of red circles 
correspond to the weight of the states in 5 layers of MnSe, closest to interface; dark and light 
blue circles denote weight of the states in the first and second QLs of Bi2Se3 near interface. The 
projected bulk bands are shown in palevioletred and rosybrown for Bi2Se3 and MnSe, respectively. 

the conduction band is used for the Bi2Se3/MnSe(lll) heterostructure calculations. It is 
important that the orthorhombic distortion of MnSe keeps the Mn magnetic moment equal 
to ±4.58/iB (it is ±4.57/iB in cubic structure with experimental lattice parameters) as well 
as antiferromagnetic ordering along the [111] direction. It means that in the direction 
perpendicular to the interface the orthorhombically distorted MnSe will provide the same 
magnetic exchange coupling as the cubic MnSe. 

The Bi2Se3/MnSe(lll) interface was studied by constructing a superlattice composed 



of Bi2Se3 and MnSe slabs. Two types of interfaces with Mn interfacial atomic layer are 
geometrically possible: fcc-type interface, where interfacial Mn atom is situated in the 
fcc-hoUow position ontop of the Bi2Se3 slab (Fig. [U^a)) and hep-type interface, where 
interfacial Mn atom is situated in the hep-hollow position (not shown). The constructed 
supercells contain two interfaces. To avoid the interface-interface interaction the slabs of 
7 quintuple layers (QLs) of Bi2Se3 and of 31(33) atomic layers of MnSe were used for the 
fcc-type (hep-type) interface. The interfacial space (separation distance between Bi2Se3 and 
MnSe slabs) as well as the atomic positions within the first (closest to the interface) QL of 
Bi2Se3 and 5 near-interface atomic layers of MnSe were optimized while interatomic distances 
within the middle part of both slabs were fixed. The total energy optimization shows that the 
fcc-type interface gains an energy of 115 meV with respect to the hep-type heterostructure. 
For this reason in the following we will focus on the fcc-type Bi2Se3/MnSe(lll) interface. 

As far as the charge transfer and charge redistribution is the common feature for any 
interface we have estimated this effect by implementing the Bader charge analysis 25|. In 
Fig. Wljo) the atomic charge difference with respect to the total electronic charge of central 
(bulk-like) atoms of Bi2Se3 and MnSe slabs is shown. In contrast to small oscillation of the 
charge in the MnSe slab and in inner QLs of Bi2Se3 a large charge redistribution is found in 
the interfacial QL. Such a change in electron charge density within the first QL results in 
substantial modification of the electrostatic potential which shows modulated band-bending 
behavior within the interfacial QL (Fig. [H^c)). Owing to the strong modification of the 1-st 
QL potential the localized states of the interfacial QL split off from the conduction band 
and spread across the gap (Fig. [2]). Additionally, two types of the states arise at -0.6 - -0.8 
eV in the local bulk energy gap: the gapped (56 meV) interfacial state and degenerate at 
the r point two spin-split states, localized in the second QL, which are split off from the 
gap edges. The latter states are similar to those which reside near the bottom of the local 



valence band gap in the TIs of Bi2Se3 family (they were studied in detail in Sb2Te3 [26|) 
but here they appear at higher energy owing to that the lower part of this gap is filled by 
the MnSe bulk states. The former, interfacial state, is similar to the state with a gap of 
~ 54 meV found in Ref. [ij], which was assumed as a gapped Dirac cone. Next interesting 
feature in the spectrum is that the topological Dirac state, being localized in the outermost 
QL on the free Bi2Se3 surface, survives in the formation of the interface relocating to the 
second QL. The small thickness of the Bi2Se3 slab (4QLs) in Ref. [ij] did not allow to catch 
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FIG. 3. (Color on-line) (a) A magnified view of electronic structure of Bi2Se3/MnSe(lll) \E{kn) 
ranges correspond to dashed (green) frame marked in Fig. [2]; (b) Spatial localization of the 
topological (red) and interfacial (blue) states at the F point, hatched area covers the Bi2Se3 layers 
with induced magnetization. 



a veritable Dirac state, relocating into the second QL. 

A magnified view of the Dirac state is shown in Fig. [3]^a). As one can see, the cone is 
gapped at the f point. Fig. |3t^b) shows that the topological state tends to leave the layers 
with induced magnetization. Thus the gap of 8.5 meV in the Dirac cone is provided by an 
overlap of the topological and spin-polarized interfacial states within the first QL (Fig.[3](b)). 
The induced magnetization at the interface is limited to three layers of Bi2Se3. The magnetic 
moment on the second layer Bi is 0.04 /iB while it is smaller on Se atoms (< 0.01 yUe). 

To elucidate the obtained results we develop an analytical model for the magnetic 
proximity effect in the Tl/Ml heterostructure, which is based on a recently proposed method 
to describe the formation of the bound in-gap electron states at the interface between 3D Tl 



and a normal insulator 



271 ]. Our model differs from an approach routinely used in the study 



of the Tl/Ml interface 13| when the existence of the exchange term in the phenomenological 
Hamiltonian of the 2D Dirac-like states is simply postulated. 



We write the full electron energy of the TI/MI heterocontact in the following form: 

n= I dre^{r)Hti-iV)e{r)+ / rfr<l>+(r)/7^(-iV)$(r) + (]/, (1) 



z>0 z<0 



nj= /"rfr[e+(r)y(r)$(r) + <l>+(r)y+(r)e(r)]. (2) 

In the TI half-space {z > 0), the four bands k ■ p Hamiltonian Hf, proposed in Ref. 28| 
for the narrow-gap semiconductors of the Bi2Se3 family, describes the low energy and long 
wavelength bulk electron states near the F point of the Brillouin zone. In the MI half-space 
(^ < 0), the electron states are modeled within the effective mass approximation by the four 
bands Hamiltonian without SOI, Hm. The intrinsic magnetization of MI is assumed to be 
perpendicular to the TI/MI interface plane {z = 0). The spinors G(r) and $(r) are smooth 
and continuous envelope functions in the right and left half-spaces, respectively. The term 
Qj involves intermixing of the TI and MI electron states at the interface via the effective 
potential of hybridization ^(r). 

Following the variational procedure |27|, one can show that the energy functional (I!])-© 
has two distinct bound states: ordinary and topological states with the exponentially 
decaying asymptotics far from the interface: $(2: -^ —00) = and 6(2; — > 00) = 0. One of 
them, the interfacial ordinary state, is localized near the interface and exponentially decays 
into the TI half-space with a scale length Zo- The charge redistribution near the interface 
causes significant shift of the ordinary state spectrum relative to the TI bulk energy spectrum 



27l |. As seen from the Boltzmann fit for AVti in Fig. [U^c), in the Bi2Se3/MnSe system, the 
band bending on the TI side of the interface is ~0.8 eV so that interfacial ordinary state is 
sunk into the region of the bulk valence band. Furthermore, due to the hybridization ([2]) 
with the orbitals of the MI outermost layer that exhibits the out-of-plane magnetization M, 
the interfacial ordinary state becomes spin polarized. 

Another state at the interface, the topological state, is not directly influenced by the 
interface potential since Qt{z = 0) = 0. This topological state is remote from the interface 
at the distance zt < Zo] note the lengths Zo and Zt are determined by the material parameters 
of TI. Thus the topological state experiences a magnetic effect of MI through the magnetic 
proximity effect, when the exchange field induced by the spin polarization of the interfacial 
ordinary state penetrates deep into the TI half-space. The magnetic proximity-induced 
gap at the Dirac point in the electron spectrum of the topological state is estimated as 
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A ~ JMS\V\'^, where J is the exchange interaction strength in TI, S ~ /g°° dz\QoQt\'^ is the 
overlap integral of the ordinary and topological states in TI. The analytical results in detail 
will be represented elsewhere. 

In summary, on the base of DFT calculations performed on the Bi2Se3/MnSe system, 
we scrutinized the magnetic proximity effect at the 3D TI/MI interface. We have shown 
that the charge redistribution and the mixing of the TI orbitals with the MI orbitals at 
the interface causes drastic modifications of the electron structure near the TI/MI interface. 
The calculation data reveal the presence of the interfacial ordinary state confined within the 
nearest interfacial QL of TI which slowly decays into MI. This state is shifted downwards to 
the local energy gap owing to the near-interface band bending. The state is gapped and spin 
polarized due to the hybridization with the MI states. On the other hand, the topological 
state is mainly localized in the second QL showing an overlap with the interfacial ordinary 
state which in this way mediates indirect exchange coupling between MI and the topological 
state. The topological state acquires the energy gap at the Dirac point proportional to the 
overlap of the topological and interfacial ordinary states. The key DFT results are supported 
by the analytical model. The unveiled mechanism of the magnetic proximity effect in the 
TI/MI structure provides a pathway to integrate TIs in spintronic devices. 
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